A Union of Professionals 

American Federation of Teachers, afl-cio 



SCHOOL VOUCHERS: 

THE RESEARCH TRACK RECORD 

STUDENT ACHIEVEMENT 





A Union of Professionals 



Edward J. McElroy, PRESIDENT 
Nat LaCour, SECRETARY-TREASURER 
Antonia Cortese, EXECUTIVE VICE PRESIDENT 



Eor further information, contact: 

Center on Accountability and Privatization 
American Eederation of Teachers 
555 New Jersey Ave. N.W. 

Washington, DC 20001 
202/879-4400 



Copyright © American Eederation of Teachers, afl-CIO (AET) 2005. Permission is hereby granted to 
APT state and local affiliates to reproduce and distribute copies of this work for nonprofit educational 
purposes, provided that copies are distributed at or below cost, and that the author, source, and 
copyright notice are included on each copy. Any distribution of such materials by third parties who 
are outside of the APT or its affiliates is prohibited without first receiving the express written 
permission of the APT. 



2 /AFT 




SCHOOL VOUCHERS: THE RESEARCH TRACK RECORD 
STUDENT ACHIEVEMENT 

A bout 15 years ago, a moribund private-school voucher movement gained new life when 
advocates started promising that vouchers would improve academic achievement, 
particularly among disadvantaged youngsters. That promise continues to be made. But now 
that there is a sizable body of evidence proving its hollowness and new legislative interest in 
“research-based” policy and practice, voucher advocates are increasingly shifting back to ideological 
arguments that can neither be proven nor disproven. Most advocates fiercely resist attempts to 
require independent evaluations of publicly funded voucher programs. And many of them now even 
minimize the importance of student achievement altogether when it comes to private school 
vouchers — but not when the subject is public school accountability, especially under the tough 
achievement standards of the federal No Child Left Behind Act (NCLB). 

But improving student achievement, particularly among disadvantaged children, continues to matter 
a great deal. It matters especially when there are competing claims on diminishing public treasuries 
and strained education budgets. Vouchers must therefore continue to be evaluated against the 
standard of student achievement. As the following summary demonstrates, research conducted from 
1990, when the nation’s first publicly funded voucher program began in Milwaukee, to the present 
fails to support the claim that vouchers improve student achievement. 

Milwaukee Parentcd Choice [Voucher] Program (MPCP, started in 1990) 

• Participating private schools are not required to test students or publicly report results as a 

condition of receiving public dollars (in 2004-05, up to $5,943 per voucher student). Pro-voucher 
legislators terminated the state’s evaluation of MPCP in 1995. Consequently, Wisconsin taxpayers 
are in the dark about the academic outcomes of a program that now enrolls 15,035 students and is 
estimated to cost more than $87 million this year alone. ^ 

• The state-mandated evaluation (1990-95) headed by John Witte of the University of Wisconsin- 
Madison found no differences in reading and math achievement between voucher students and 
their public school peers. ^ 

• Harvard University’s Paul Peterson, a voucher advocate, and his team reanalyzed the state’s 
MPCP evaluation and claimed to find that by the third and fourth year of attending a private 
school, voucher users outperformed the public school control group in reading and math. The 
researchers’ own tables show a statistically significant result only in math and only in the fourth 
year. Moreover, their data showed that voucher students who were still in MPCP by that time 
were a more advantaged group than the comparison students in public schools.^ 

• A second reanalysis of the state’s MPCP evaluation by Princeton University’s Cecilia Rouse found 
no effect from vouchers in reading and a small effect in math for MPCP students who had 
remained in the program over the four-year period. Rouse also found that the large number of 
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dropouts (or pushouts) from MPCP were primarily students who had been performing poorly in 
the private schools/ 

• A follow-up study hy Rouse found that low- income students attending Milwaukee public schools 
served by a state class-size reduction and enrichment program significantly outperformed 
voucher students in reading and scored as well in math/ 

Cleveland Scholarship [Voucher] and Tutoring Program (CSTP, started in 1996) 

• In 2004-05, CSTP enrolled an estimated 5,675 students at a budgeted cost of $17,901,887. The 
lion’s share of the cost for CSTP has been funded by reducing the Cleveland public school 
district’s amount of state Disadvantaged Pupil Impact Aid by $11,901,887.*^ The state of Ohio 
contracted with a team of researchers led by Kim Metcalf of Indiana University to evaluate CSTP. 
The most recent results of their longitudinal study (1998-2003) were released in October 2004.^ 

• The Metcalf evaluation found no achievement differences in five out of six subjects — and no 
difference in overall achievement — between voucher students who had attended private schools 
from kindergarten through grade 5 and the two public school comparison groups (applied for 
vouchers but did not receive, did not apply for vouchers).** These results are particularly striking 
because although Metcalf controlled for race, he did not account for family income differences 
between the public and private school students. 

• At the beginningoi first grade, students who had entered CSTP as kindergartners had significantly 
higher achievement than the public school comparison groups (no data are provided on whether 
these students had attended kindergarten) . By the end of the first grade, the public school groups 
had closed that gap.** 

• Overall, students in the public school comparison groups made greater gains from first through 
fifth grade than the voucher students, even though the voucher students were less likely to be 
African-American and low income. 

Florida Opportunity Scholarship [Voucher] Program (started in 1999; twice struck down as 

unconstitutional by state courts, but operating while on appeal to the state’s Supreme Court) 

• Any student enrolled in a public school that is deemed “failing” for two of the previous four years 
under the state’s testing and accountability program (A-Plus) is eligible for a private school 
voucher equal to the per-pupil public school expenditure or the total tuition and fees of the 
private school, whichever is less. The Florida legislature has never provided for an evaluation of 
the program." 

• In 2001, a study by voucher proponent Jay Greene of the Manhattan Institute, a voucher advocacy 
group, found that the mere threat of losing students to vouchers caused public schools rated as 
failing in 1999 to improve." To test Greene’s claim, Doug Harris of the Economic Policy Institute 
analyzed achievement in Florida’s schools before and after the threat of vouchers was introduced. 
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There were no clear differences in the improvement of low- achieving (or other) schools pre- and 
post-vouchers, which makes Greene’s claim insupportable. 

• A reanalysis of Greene’s study hy Gregory Gamilli and Katrina Bulkley, both of Rutgers University, 
found that his methods produced greatly inflated estimates of the gains in the voucher- 
threatened “failing” schools and that his results failed to account for the effect of the substantial 
new resources that had been put into those schools.''* 

• In 2003, Greene and Marcus Winters updated the 200 1 Greene study and again found that a 
voucher threat improves low-achieving schools, now adding that improvements were “in direct 
proportion” to the degree of that threat."* Differences among schools and student characteristics 
were only partially accounted for in the analysis, nor did the study consider the impact on schools 
of the threat of NGLB sanctions. 

Florida’s John M. McKay Scholarship [Vouchers] for Students with Disabilities Program (started 

statewide in 2001) 

• Florida does not require student testing or public reporting of achievement for participating 
private schools, which do not have to employ any special education teachers. Nor has the state 
provided for an evaluation. There currently are about 13,700 students receiving McKay vouchers, 
which cost taxpayers $72,314,026 in 2004-05. Parents who accept a voucher waive rights under the 
Individuals with Disabilities Education Act (IDEA).*'* 

D.C. Choice [Voucher] Incentive Program (enacted by Congress in 2004 and targeting the District of 

Columbia, this is the first federally funded voucher program in the nation) 

• This five-year, $14 million voucher “demonstration” project is targeted to low-income students, 
with priority access for students attending D.C. public schools that are identified under NGLB as 
needing improvement. The maximum amount of the voucher, which is awarded through an 
applicant lottery, is $7,500. Participating private schools can select voucher students using their 
regular admissions procedures and can charge parents additional tuition and fees above the 
voucher amount. Although the law does not require the private schools to test students or 
publicly report achievement results. Congress did provide for an evaluation of the program, 
beginning in its initial operating year (2004-05) . The U.S. Department of Education awarded the 
evaluation contract to Westat, an independent evaluation firm, and Georgetown University’s 
Patrick Wolf, who is considered a voucher advocate. 

• The voucher law mandates an evaluation that compares the achievement of students who won 
and used a voucher with students who lost in the voucher lottery. But too few eligible students 
applied for the 1,613 vouchers, leaving too few “losers” for the evaluation to be run in the 
mandated way, at least in the important baseline year. Of the 1,359 students who won the voucher 
lottery, at least 290 (21.3 percent) did not use the voucher.'^ 
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• Although Congress mandated that students in public schools identified hy NCLB as needing 
improvement should get priority in receiving vouchers, 15.4 percent of voucher students 
(prekindergarten and kindergarten) had never even attended school before, and 19.4 percent had 
attended a private school the year before. Only 74, or fewer than 6 percent, of the 1,359 voucher 
lottery winners came from public schools that fit Congress’s “priority.” How many of these 74 
students were among the 1,013 voucher awardees who actually gained admission to a private 
school is unknown.^’' 

Privately Financed Voucher Programs 

• Among the numerous studies of non-publicly funded voucher programs, the U.S. Government 
Accountability Office (GAO)^^ identified only three that were “rigorous enough to meet our criteria 
for inclusion.” Each of these studies of programs in Washington, D.C., Dayton, Ohio, and New 
York City included voucher advocate Paul Peterson on the research team.^“ 

• Both in their reports and in the press, the Peterson research teams claimed to find large 
achievement gains for African-American voucher students, but not for whites or Hispanics. The 
GAO review concurred that vouchers were ineffective for whites and Hispanics in all three cities; it 
also found that in Washington, D.C. and Dayton, there was no difference in the achievement of 
African-American voucher users and that of the public school comparison group either.^’ 

• The GAO validated only the New York City study that showed achievement gains for low- income 
African-American students who used vouchers while in grades 1-4. However, the GAO also called 
attention to a then-unpublished analysis of the New York study by Alan Krueger and Pei Zhu of 
Princeton University, noting that “Their findings raise doubts about the size and significance of 
earlier findings of a positive effect of vouchers on test scores for African American students. 

• Krueger and Zhu’s analysis of the New York voucher study appeared in January 2003, about six 
months after the GAO report. Among other problems, they found that the Peterson team had 
excluded scores for about 40 percent of the students in the sample and had used a novel 
classification of student race. When the data were corrected, the Peterson team’s much-vaunted 
“miracle” results for African-American voucher students in New York disappeared. “The safest 
conclusion,” observed Krueger and Zhu, “is probably that the provision of vouchers did not lower 
the scores of African American students. 

• Krueger and Diane M. Whitmore compared the Peterson team’s uncorrected, positive voucher 
results for African-American students to the results obtained for African-American students in the 
Tennessee class-size reduction experiment, which is considered the most rigorous experiment 
ever conducted in education. Even using Peterson’s inflated results, Krueger and Whitmore found 
that black students in small classes for two years “improved their test performance by around 50 
percent more than the gain experienced by black students who attended a private school as a 
result of receiving a voucher in the New York, Dayton and D.G. voucher experiments.”^'* 
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